Papers with interpolation strategies
Discretized Integrated Gradients for Explaining Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Integrated Gradients (IG) is widely adopted due to its desirable explanation axioms and the ease of gradient computation. |
| Approach: | They propose an attribution-based explanation algorithm that uses averaging the model's output gradient interpolated along a straight-line path in the input data space. |
| Outcome: | The proposed method is compared with IG on multiple sentiment classification datasets. |
kNN-LM Does Not Improve Open-ended Text Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Interpolation-based retrieval-augmented language models (LMs) are a subtype of retrieval augmented language model that computes the probability of the next token by interpolating between the softmax distribution of the original LM and a token distribution formed by retrieving over an external datastore. |
| Approach: | They propose to interpolate the predicted distribution of the next word with a distribution formed from the most relevant retrievals for a given prefix. |
| Outcome: | The proposed methods do not exhibit improvements in open-ended generation quality, as measured by automatic evaluation metrics and human evaluations. |